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Abstract 

This study examined the effects of a research-based intervention, schema-based instruction 
(SBI), implemented by experienced- (taught SBI in previous study; Jitendra et ah, 2015) and 
novice-teacher implementers (taught SBI for the first time with professional development) on the 
mathematics outcomes of seventh-grade students. SBI is a multicomponent intervention that 
emphasizes the mathematical structure of problems through the use of schematic diagrams and 
incorporates problem solving and metacognitive strategy instruction. Results indicated that both 
experienced- and novice-teacher implementers delivered SBI with similar levels of fidelity; there 
was no SBI experience effect on the immediate and 10-week retention tests of proportional 
problem-solving, on a general measure of problem solving, or on the end of the year state 
mathematics achievement test. These results provide evidence that the effectiveness of SBI 
generalizes over time to different cohorts of teachers and that the impact of SBI on student 
mathematics outcomes is maintained over time without additional PD. 

KEYWORDS: schema-based instruction, experienced and novice implementers, seventh-grade 
students, proportional problem solving 
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Schema-Based Instruction: Effects of Experienced and Novice Teacher 
Implemented on Seventh Grade Students’ Proportional Problem Solving 
1. Introduction 

Ratio and proportional relationships are of primary importance during the upper 
elementary and middle school grades. These relationships, along with the interrelated topics of 
fractions, decimals, and percent, provide a critical foundation for algebra. Proportionality 
involves the concept of ratio and is central to topics in mathematics such as linear functions, 
scale drawings, similarity, trigonometry, and probability. In the Common Core State Standards 
(CCSS; National Governors Association Center for Best Practices & Council of Chief State 
School Officers, 2010), instructional time focused on proportionality occurs in middle school 
when students “develop understanding of proportionality to solve single and multi-step problems 
... solve a wide variety of percent problems, including those involving discounts, interest, taxes, 
tips, and percent increase or decrease” (p. 46). 

Researchers in mathematics education (e.g., Lamon, 2007; Lobato, Ellis, Charles, & 
Zbiek, 2010) note that ratio and proportional relationships are situated in the broader landscape 
of the multiplicative conceptual field (e.g., whole number multiplication and division, fractions, 
linear functions). While young students’ difficulty with ratio and proportional relationships is 
related to their development of multiplicative versus additive reasoning (Lamon, 1995), 
secondary school students who learn to reason proportionally following instruction in various 
linear functions tend to apply the notion of linearity to most situations, even when it is not 
applicable (see Fernandez, Llinares, Van Dooren, De Bock, & Verschaffel, 2012; Van Dooren, 
De Bock, Hessels, Janssens, & Verschaffel, 2005). 


3 



PROPORTIONAL PROBLEM SOLVING 


Solving even simple proportion problems is challenging for many children and 
adolescents as they may not understand the problem situation or know when a solution strategy 
is applicable (Weinberg, 2002). Yet, only few intervention studies have focused on improving 
students' learning of ratios and proportions. Most studies were short-term and did not address the 
broad domain of ratios and proportional relationships (Adjiage & Pluvinage, 2007; Fujimura, 
2001; Miyakawa & Winslow, 2009) or used quasi-experimental research designs or teaching 
experiments, which limited causal inferences. Also, few studies have tested the effectiveness of a 
comprehensive curriculum package (e.g., Connected Mathematics Project; see Ben-Chaim, 
Fitzgerald, Benedetto, & Miller, 1998) or conducted randomized studies. The few randomized 
studies conducted have examined the efficacy of schema-based instruction (SBI), a 
multicomponent approach to teaching proportional problem solving (e.g., Jitendra, Star, Dupuis, 
& Rodriguez, 2013; Jitendra et ah, 2015). 

In this article, we report findings from the second year of a 2-year study of the efficacy 
and sustainability of SBI. In the first experimental study (Jitendra et ah, 2015), teachers were 
randomly assigned to either the treatment condition (Cohort 1), in which they received 
professional development to implement SBI five days a week for approximately 6 weeks to teach 
problem solving involving ratio, proportion, and percent, or to the control condition (Cohort 2) to 
teach the same topics from their district-adopted mathematics textbook. Results indicated that 
students in the SBI classrooms outperformed students in the control classrooms on a proportional 
problem solving measure and maintained their improved performance nine weeks later. Scores 
on the Process and Application subtest of the Group Mathematics Assessment and Diagnostic 
Evaluation (GMADE) were equivalent for the two groups. 
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Many teachers, even though highly motivated, may find it challenging to use new 
instructional materials to teach ratios and proportional relationships and effectively use visual 
representations to prime the underlying problem structure, foster problem solving and 
metacognitive strategy skills, and develop procedural flexibility (essential features of SBI 
implementation). Implementing a new instructional approach can pose a range of challenges 
even when teachers are provided with instructional materials (Obara & Sloan, 2010; Remillard, 
2005). Factors such as initial mismatch between principles underlying the innovative approach 
and teachers’ philosophy of teaching and learning, time to cover the rest of the curriculum, and 
the methods of assessment may restrict innovation that attempts to find a balance between (a) 
providing instruction to ensure meaningful learning and (b) having teachers “reflect on the 
mathematical value of the study of the topic” (Bennie & Newstead, 1999, p. 5). Despite these 
challenges, there is some research suggesting that instructional materials in conjunction with 
professional development can change teacher practices to align with the innovation (Cohen & 
Hill, 2000). 

In response to the challenges of using new instructional materials, we invited all teachers 
who participated in the first study (Year 1; Jitendra et al., 2015) to continue in the study a second 
year to evaluate the effects of professional development (PD) and sustainability of SBI by 
examining both teacher implementation and student learning in classrooms of teachers who 
received PD and implemented SBI in the past year and possibly gained more expertise with this 
method (experienced-teacher implementers) and teachers implementing SBI for the first time 
with PD (novice-teacher implementers). Previous research on the long- and short-term effects of 
teachers' professional development is sparse, with one study (Fennema et al., 1996) documenting 
that when teachers participate in PD and develop expertise with instructional techniques for 
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promoting students’ mathematical thinking, greater gains are seen in subsequent cohorts of 
students than prior cohorts. Therefore, the purpose of this study was to examine whether teacher 
participation in PD along with more or less experience with SBI would differentially affect 
teacher implementation and influence their students’ proportional and mathematical problem 
solving skills as well as mathematics achievement. 

1.1 Theoretical Framework 

1.1.1. Teacher learning in organized professional development. Research has shown that 
high quality professional development can not only affect teacher attitudes and classroom 
practice, but also influence student learning (Garet, Porter, Desimone, Binnan, & Yoon, 2001). 
Several conceptual frameworks of professional development have been proposed for enhancing 
teacher and student learning (e.g., Borko, 2004; Desimone, 2009). One theory involves a link 
between the PD, teachers, and students such that teachers participate in PD, they implement the 
practices learned in PD in their classrooms, and these practices affect student achievement 
(Desimone, 2009). Features of high-quality PD that have been associated with changes in teacher 
practice and subsequent improvements in student perfonnance include content focus, active 
learning, coherence, duration, and collective participation. A sustained focus on content appears 
to be a critical feature of high-quality PD (Blank & de las Alas, 2009). Content-focused PD 
activities designed to help teachers understand what they teach and how students leam the 
content can support teacher knowledge and practice to improve student learning (Sample 
McMeeking et ah, 2012; Perry & Lewis, 2011; Saxe, Gearhart, & Nasir, 2001). 

There is evidence that teachers engaged in active learning strategies through observation, 
discussion, practice, and reflection are particularly effective (Desimone, Porter, Garet, Yoon, & 
Binnan, 2002). Such strategies may include implementing new approaches to teach familiar 
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content that support teachers’ thinking and reflecting on mathematical ideas or helping teachers 
understand students’ difficulties in specific content by reviewing student work (Carpenter, 
Fennema, Peterson, & Loef, 1989; Cohen & Hill, 2000). One exemplary PD program that not 
only supports teachers’ own knowledge of the relevant mathematics, but also enhances their 
understanding of children’s mathematical understanding is Cognitively Guided Instruction 
(CGI). Research on CGI, which focused extensively on instructional practices, showed that the 
4-week PD program led to changes in classroom teaching practices (e.g., fostered discussions of 
problem-solving strategies) and subsequent improvement in children’s mathematical word 
problem solving performance. Students in CGI classrooms solved a variety of mathematics 
problems using multiple-solution strategies and demonstrated confidence in their mathematical 
ability compared to students in control classrooms. Furthermore, findings of a longitudinal study 
of CGI (Fennema et ah, 1996) indicated that CGI teachers developed their own practices as they 
gained expertise with CGI approaches over time, which led to further improvements in 
subsequent cohorts of students. 

Another core feature of effective PD programs is coherence, which refers to the extent to 
which the content taught aligns with state standards and assessments, is consistent with teacher 
goals for their professional development, and presents opportunities for communicating with 
other professionals (Garet et al. 2001; Penuel, Fishman, Yamaguchi, & Gallagher, 2007). 
Effective PD also requires considerable time that is carefully structured, organized, and focused 
on both content and pedagogy (Guskey & Yoon, 2009). Content-focused PD along with 
extended duration (more than 30 hours over a span of time) can lead to changes to teacher 
practice and student achievement (see Blank & de las Alas, 2009; Desimone et al., 2002; Guskey 
& Yoon, 2009; Yoon, Duncan, Lee, Scarloss, & Shapeley, 2007). Furthermore, collective 
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participation of groups of teachers working together is more likely to influence teacher learning 
(see Desimone et ah, 2002; Penuel et ah, 2007; Penuel, Sun, Frank, & Gallagher 2012). 
Although there is some evidence that PD with all or most of these characteristics can support 
better curriculum implementation (Perry & Lewis, 2011) and enhanced student learning (Sample 
McMeeking et ah, 2012; Perry & Lewis, 2011; Saxe et ah, 2001), there is very limited causal 
evidence of the effects of teacher PD on improving student mathematics achievement (Gersten, 
Taylor, Keys, Rolfhus, & Newman-Gonchar, 2014). 

Guided by findings from prior research on classroom practice and student learning, we 
designed our professional development program to focus on the SBI curriculum content on 
proportional reasoning as the basis for teachers’ understanding of proportional reasoning and of 
the ways that students understand and learn. The PD was organized to provide teachers an 
opportunity to participate with other teachers in practices reflective of state standards (e.g., make 
sense of problems, look for and make use of structure, model with mathematics). The goal of the 
PD was to enhance effective teacher practices through observation, discussion, practice, and 
reflection. For example, an introductory activity that emphasized promoting deeper knowledge 
of the content of the curriculum required teachers to work in pairs or small groups to propose 
strategies for solving a non-routine problem that had insufficient infonnation followed by the 
facilitator engaging teachers in reflection of ratios and proportional relationships and 
understanding students’ efforts to solve such problems. 

1.1.2. Schema-Based Instruction. SBI has its roots in schema theory of cognitive 
psychology, research on expert problem solvers, and is guided by cognitive models of 
mathematical problem solving. Consistent with schema theory, recognition of the semantic 
structure of the mathematical problem is critical to problem comprehension (Kalyuga, 2006) and 
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is an essential feature of SBI. SBI integrates essential processes that expert problem solvers 
engage in when solving problems such as “perceiving rapidly and accurately the mathematical 
structure of problems and in generalizing across a wider range of mathematically similar 
problems” (Van Dooren, de Bock, Vleugels, & Verschaffel, 2010, p. 22). Providing students 
with problem categories and ways to place problems within categories improves problem solving 
(Chen, 1999; Quilici & Mayer, 1996). 

In addition, SBI emphasizes knowledge of problem solving procedures (e.g., problem 
representation, planning) for a given class of problems (see Marshall, 1990; Mayer, 1999). 
Teachers are guided to use effective instructional practices (e.g., explicit modeling, guided 
questions to engage students) to help students recognize the problem type, represent the problem 
using a diagram, solve the problem, and check the solution. Another feature of SBI is the focus 
on procedural flexibility by having students use, share, compare, and contrast multiple solution 
methods for a given class of problems to improve procedural knowledge (Star & Rittle-Johnson, 
2009). 

Also integral to SBI is teaching students to display metacognition skills similar to expert 
problem solvers, who are able to plan, check, monitor, and evaluate their perfonnance (e.g., 
Kramarski & Mevarech, 2003; Mevarech, & Kramarski, 2003). Guided by their teachers, 
students “think about what they are doing and why they are doing it, evaluate the steps they are 
taking to solve the problem, and connect new concepts to what they already know” (Woodward 
et ah, 2012, p. 17). 

1.1.3. Previous Research on SBI and Proportional Reasoning. Jitendra and colleagues 
focused on the broad landscape of proportional problem solving, which included a set of 
mathematical topics (e.g., ratio, proportion, percent, percent of change, simple interest, scale 
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factor) whose boundaries are not easily identified. In the first study, Jitendra et al. (2009) 
randomly assigned seventh-grade classrooms to SBI or a “business as usual” control condition. 
Students in both conditions were taught the same topics (i.e., ratios, equivalent ratios, ratio word 
problems, rates, proportion word problems, scale drawing problems) for the same time (daily, 45 
min sessions for two weeks). SBI students scored on average significantly higher than 
comparison students on proportional problem solving (d = 0.45) and maintained the effects on a 
4-month retention test (d = 0.56). 

Jitendra and colleagues conducted two randomized controlled studies to address several 
limitations of the Jitendra et al. (2009) study. These studies targeted additional topics (e.g., 
proportional problem solving involving percent), included more classrooms and schools, 
extended instructional time to 6 weeks, provided longer professional development (Jitendra et 
al., 2013; Jitendra, Star, Rodriguez, Lindell, & Someki et al., 2011), and reduced direct 
involvement of the research team in SBI implementation (Jitendra et al., 2013). Classrooms were 
randomly assigned to SBI intervention or control (business as usual) conditions, and teachers 
with multiple classrooms taught both SBI and control classrooms all of which were included in 
the studies. Results indicated that SBI students in both studies (Jitendra et al., 2011, 2013) 
outperformed comparison students on a measure of proportional problem solving at posttest 
(multilevel standardized effect sizes of 0.32 and 0.36). The effects of SBI were not maintained 
on the retention test given 4 weeks after the end of the intervention in Jitendra et al. (2011), 
which the authors attributed to a lack of power to detect significant differences given the modest 
number of classrooms (n = 21). Jitendra et al. (2013) included more classrooms (n = 42) and 
showed that SBI students maintained their problem solving skills at 6-week follow-up 
(multilevel standardized effect size = 0.29). 
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Related specifically to the study reported in this manuscript is a previous randomized 
study that improved upon earlier study designs by randomly assigning teachers to SBI or control 
conditions and then randomly selecting one of their classrooms to participate in the study such 
that each teacher taught in a SBI or control classroom but not both (Jitendra et ah, 2015). 
Furthermore, the study assessed whether the efficacy of SBI would hold when comparison 
classrooms used increasingly diverse mathematics programs and when we eliminated direct 
involvement of the research team to support SBI classroom implementation and test 
administration. Results indicated that students in SBI classrooms on average outperformed 
students in control classrooms on a measure of proportional problem solving at posttest 
(multilevel standardized effect size = 0.46) and maintained their problem solving skills at 9 
weeks follow-up (multilevel standardized effect size = 0.32). While our previous SBI studies 
indicate the promise of our intervention when teachers implement SBI immediately after 
receiving training on its use, it is less clear whether teachers will persist in implementing SBI 
with fidelity in subsequent years. Based on two previous SBI studies (Jitendra et ah, 2011, 2013), 
we found that teachers who are experienced SBI implemented (implemented SBI in both 
studies) gained expertise with SBI as a result of using our instructional materials to seamlessly 
implement SBI and continued to incorporate instructional practices from the PD training into 
their teaching (e.g., providing prompts that require students to think through the problem solving 
process and justify their responses). However, we did not test the relative effects pertaining to 
more or less experience with SBI on student learning. 

1.3. The Current Study 

The present study extends previous SBI research by including the same sample of 
teachers as in Jitendra et al. (2015) as a means to determine the relative effects of SBI with 
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experienced-teacher implemented (Cohort 1 - treatment teachers participated in our professional 
development and implemented SBI in the previous study) and novice-teacher implemented 
(Cohort 2 - control teachers from the previous study participated in PD and implemented SBI for 
the first time in this study) on the proportional and mathematical problem solving performance of 
a new group of seventh-grade students. In addition, this study examined students’ and teachers’ 
attitudes in terms of acceptability and benefits of SBI. To our knowledge, the study reported here 
is the first research study to examine whether the effects of PD and teachers’ expertise or 
experience with SBI leads to better implementation and subsequent improvement in student 
achievement compared to having no prior experience with SBI and implementing for the first 
time following PD proximal to treatment enactment. Specifically, we investigated two 
relationships: (a) between PD and teacher practice and (b) between teacher practice and student 
achievement. We considered the following research questions and hypotheses: 

1. What is the effect of the PD training provided in this study for teachers who have prior 
experience with SBI vs. teachers with no prior experience with SBI on their classroom 
instructional practice in terms of fidelity of treatment implementation? Given the limited 
research on the short- and long-tenn effects of teachers’ professional development, it is not clear 
a priori whether Cohort 1 teachers do a better job of implementing SBI in their second year of 
the study (despite not receiving any additional professional development in the second year) or 
whether the novice SBI teacher implementers, who have had a more proximal professional 
development experience, may implement SBI with a higher degree of fidelity than Cohort 1 
teachers. 

2. Did the students of teachers who participated in the PD training and had prior 
experience with SBI: (a) demonstrate higher levels of proportional problem solving performance 
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on an immediate posttest and on a retention test (10 weeks later) than did the students of teachers 
who participated in the PD program and had no prior experience with SBI? (b) demonstrate 
higher levels of overall mathematical problem solving performance and mathematics 
achievement as measured by standardized assessments than did the students of teachers who 
participated in the PD program and had no prior experience with SBI? Similar to research 
question 1, it is not clear a priori whether students of Cohort 1 teachers would score higher than 
or comparably to students of Cohort 2 teachers on study outcomes. On the one hand, it may be 
the case that students of experienced SBI teacher implementers (translates to better SBI 
implementation) outperform students of novice SBI teacher implementers. On the other hand, the 
novice SBI teacher implementers, with more proximal professional development experience, 
may better implement SBI that would lead to greater student learning. 

3. Did students and teachers who participated in the study report different levels of 
satisfaction with SBI based on their assignment to condition? On the basis of prior research on 
teacher participation in PD programs designed to help understand children’s mathematical 
thinking (e.g., Carpenter et al. 1989) and previous research on SBI (e.g., Jitendra, Harwell, 
Dupuis, & Karl, in press), it is hypothesized that students’ and teachers’ beliefs about SBI 
program would be positive, but the direction of those effects is not clear. 

A priori, it is not clear whether the students of Cohort 1 teachers score higher or similarly 
to students of Cohort 2 teachers on study outcomes. On the one hand, it may be the case that 
teachers do a better job of implementing SBI in their second year of the study (despite not 
receiving any additional professional development in the second year), in which case the students 
of experienced SBI teacher implementers may outperform the students of novice SBI teacher 
implementers. On the other hand, the novice SBI teacher implementers will have had a more 
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proximal professional development experience and thus may implement SBI with a higher 
degree of fidelity, which in turn may lead to greater student learning. 

2. Method 

2.1. Participants and Sampling Procedures 

2.1.1. Teachers. Sixty-nine seventh-grade public school teachers who participated in the 
SBI study in the previous year (Jitendra et ah, 2015) returned in Year 2. A total of 13 teachers, 
nine from Cohort 1 (treatment) and four from Cohort 2 (control), did not continue after Year 1 
due to changes in teaching assignments in that a teacher no longer taught a typical seventh-grade 
mathematics class and therefore was not eligible, or due to retirement or family circumstances. 
Because Cohort 2 teachers taught SBI for the first time during Year 2, we recruited four new 
teachers from the same schools to replace the four Cohort 2 teachers who did not continue. 
Thus, the current study included 73 teachers, 31 in Cohort 1 and 42 in Cohort 2. 

Teachers’ mean years of experience teaching mathematics was 12.2 years (SD = 6.7, 
range 1 to 34 years). The majority of teachers were White in = 66), while four teachers were 
Hispanic, two teachers were Asian, and one teacher was American Indian. Sixty-three percent of 
teachers were female. 

2.1.2. Students. The sample consisted of seventh-grade students in 48 school districts (36 
rural, eight suburban, and four urban) in an upper Midwest state in the U.S. At the beginning of 
the school year each teacher identified the class periods during which typical math instruction 
would be delivered, and one of these classes was randomly selected to participate in the current 
study. All students in the participating class that were present at the pretest and posttest were 
included in the student sample. The original student sample consisted of 1,859 seventh-grade 
students; the majority of students were White (77%), with 8% Black, 8% Hispanic, 6% Asian, 
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and 1% American Indian. Due to data analysis difficulties li nk ed to small sample sizes American 
Indian students (n = 14) were removed; as such, the student sample used for the present analyses 
consisted of 1,845 students (834 in Cohort 1 and 1,011 in Cohort 2). The mean age of these 
students was 12 years 6 months (SD = 4 months). Approximately 40% of students were eligible 
for a free or reduced price lunch (FRL), 10% received special education services, and 6% were 
English language learners (ELL) (see Table 1 for teacher and student demographic information). 

2.2. Research Design 

Our focus on comparing student performances in Year 2 of the study for Cohort 1 and 
Cohort 2 prompted us to adopt a nonequivalent, quasi-experimental, cluster design (Cohort 1 vs. 
Cohort 2) in which students were treated as nested within clusters (teachers). The precise nature 
and magnitude of the selection bias associated with the quasi-experimental design is unclear. 
Following the What Works Clearinghouse (WWC, 2014) standards, control variables were used 
in the data analyses to take pre-existing differences among cohorts into account. This helped to 
ensure, but does not guarantee, credible inferences about the impact of Cohorts 1 and 2 on 
student proportional problem solving learning. 

2.3. Procedures 

2.3.1. Professional development. The 42 Cohort 2 teachers participated in professional 
development (PD) in the current study year that included training in SBI over 2 days (8 hr per 
day) on implementing the treatment practices. Cohort 1 teachers participated in a similar PD 
during the previous year and did not receive additional PD prior to implementing SBI for the 
second time during Year 2 of the study. As such, both cohorts of teachers received PD prior to 
implementing SBI for the first time. 


15 



PROPORTIONAL PROBLEM SOLVING 


In both years of the study, the PD was designed to support teachers’ learning and 
implementation of SBI. The PD focused on critical features of the SBI practices (e.g., 
recognizing problem types, generating estimates, applying multiple-solution strategies) to 
support student learning of ratio, proportion and percent, as well as how to teach students to 
apply these practices. Teachers were provided with SBI program materials for instruction and 
were given the opportunity to review the two curricular units (ratios/rates and percent), view 
multiple short video segments of other teachers implementing the SBI intervention, and discuss 
how to develop students’ proportional reasoning skills. Although lessons were scripted to 
provide a detailed teaching procedure (i.e., questions to ask, examples to present), teachers were 
asked not to read the scripts verbatim. Video segments of SBI teachers in previous studies 
highlighted the importance of using their own explanations and elaborations to implement SBI. 

2.3.2. Description of the SBI intervention. The SBI program consists of two units, the 
first focusing on Ratio/Proportion (e.g., equivalent ratios, rates, ratio and proportion word 
problem solving, scale drawings) and the second on Percent (e.g., percent increase or decrease, 
including those involving discounts, interest, taxes, tips). The program comprises 21 lessons, 
which cover the CCSS content standards of ratios and proportional relationships (For further 
details of the SBI instructional approach, see Jitendra et ah, 2009, 2011, 2013, 2015). Teachers 
taught the two units using our SBI materials daily for 50 min a day for approximately six weeks 
during their regularly scheduled seventh-grade mathematics class. 

The SBI intervention for solving proportional word problems consists of four 
instructional practices: (a) model problem solving and metacognitive strategies, (b) activate the 
mathematical structure of problems, (c) visually map infonnation in the problem using schematic 
diagrams (see Appendix A for ratio, proportion, and percent of change diagrams), and (d) 
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develop procedural flexibility. The problem solving and metacognitive strategy instruction 
components allow students to engage in applying learned content (e.g., ratios/rates, percent) in 
problem solving activities (i.e., recognizing the problem type, identifying and representing 
critical infonnation in the problem using an appropriate diagram, connecting the problem to what 
is already known, estimating the answer, selecting a strategy to solve the problem, solving the 
problem and presenting the solution within the context of the problem, checking the 
reasonableness of the solution) and metacognitive activities (i.e., monitoring and reflecting on 
the problem-solving process). Teachers modeled these four practices through an extensive use of 
think-alouds with time for students to work with a peer and independently to apply each practice. 
Furthermore, teacher-student dialogues helped to clarify and refine student thinking as they 
solved problems. 

Students were taught a problem solving strategy to solve proportion word problems. First, 
teachers provided opportunities for students to engage in whole class discussions to identify the 
type of problem (ratio, proportion, or percent) by reading, retelling, and examining information 
in the problem as well as thinking about how problems within and across types are similar or 
different. Second, students learned to connect the problem to a certain schematic diagram and 
represent the problem such that the visual mapping showed the relevant elements, relations and 
conditions embedded in the problem critical to solving the problem. Third, students were 
instructed to estimate an answer, make decisions about what solution method (equivalent 
fractions, unit rate, cross multiplication) to use (fourth practice), and solve the problem. The goal 
of this strategy is to develop procedural flexibility by having students become cognizant of the 
various methods and select the method that is most efficient based on the numbers in the 
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problem. Finally, in checking the solution, students learned to use their estimated answer from 
the previous step in evaluating their work to determine whether the answer made sense. 

Teachers used prompts or deep-level questions to support student learning and promote 
metacognition by having them monitor and reflect on the problem solving process. Each 
problem-solving step was accompanied by questions to ensure that students (a) understand and 
identify the problem type (e.g., Why is this a proportion problem? How is this problem similar to 
or different from one I already solved?), (b) identity and represent the critical information in the 
problem using an appropriate diagram (e.g., Which diagram is best to represent infonnation in 
the problem?), (c) select a strategy to solve the problem (e.g., Which solution method would best 
help me solve this problem?), and (d) check the solution (e.g., Is the answer reasonable based on 
my estimate?). 

2.3.3. Fidelity of implementation. We assessed procedural fidelity and adherence to the 
SBI intervention by coding classroom videos using a rubric. The rubric focused on procedural 
fidelity and generated scores ranging from 0 (did not implement) to 3 (highly aligned with SBI 
practices). For all teachers we coded videos of a lesson on proportion problem solving as a whole 
for seven features that represent essential problem solving activities in SBI: (a) identifies the 
problem type by focusing on the key problem features, (b) connects the new problem to 
previously solved problems, (c) represents critical information in the problem text using an 
appropriate diagram, (d) generates an estimate prior to solving the problem, (e) discusses 
multiple solution strategies, (f) solves the problem and presents the solution within the context of 
the problem, and (g) evaluates the solution. 

Following coder training on use of the rubric, two raters independently assessed fidelity 
for each classroom video, producing a total of two raters x 73 teachers = 146 ratings (i.e., two per 
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teacher). Disagreements in ratings were resolved through discussion and review of the 
videotapes. We estimated inter-rater reliability by computing intra-class correlations for the 
ratings, which averaged 0.84 (range 0.64 to 0.96). 

2.4. Measures 

Teachers administered the proportional problem-solving (PPS) test and Group 
Mathematics Assessment and Diagnostic Evaluation (GMADE) test prior to and immediately 
following the SBI intervention, with the PPS test also administered 10 weeks following 
treatment. We collected students’ sixth- and seventh-grade scores on the Mathematics Minnesota 
Comprehensive Assessment-Series III (MCA-III), which is a state-mandated assessment 
(Minnesota Department of Education, 2014), as well as student race, sex, special education, and 
ELL and FRL status. Students and teachers also completed surveys about their satisfaction with 
the treatment at the end of the study. 

2.5.1. PPS test. This assessment, used in prior studies (e.g., Jitendra et ah, 2015), 
consisted of 22 multiple-choice questions and four short-response items testing students’ 
proportional problem solving performance (see Appendix A for a sample short answer item). 
Each multiple-choice question was dichotomously scored, and all multiple-choice items were 
machine scored. The research team was blind to the study conditions and scored the four short- 
response items using a rubric (e.g., using sample student responses taken from a previous study), 
which emphasized correct reasoning. Responses were scored on a 0-to-2 point scale (Appendix B 
presents the scoring rubric for the sample short answer item). Inter-rater reliability for the short 
answer items was estimated using an intra-class correlation and averaged 0.92, 0.93, and 0.90 at 
pretest, posttest, and delayed posttest, respectively. 
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We calculated students’ scores on the PPS assessment by taking the sum of their total 
points earned (total possible points equaled 30), which means that the short-response and 
multiple-choice items contributed unequally to the overall score. To assess the internal 
consistency of the PPS we used the jMetrik software (Version 2.1.0; Meyer, 2011) to fit a 
congeneric model assuming a single continuous latent factor underlies the dichotomous- and 
trichotomously-scored PPS items (McDonald, 1999). The coefficient omega (Dunn, Baguley, & 
Brunsden, 2013) values for the PPS pretest, posttest, and delayed posttest of 0.75, 0.81, and 0.83, 
respectively, represent reliabilities estimated as the ratio of true score variance to observed score 
variance (Dunn et ah, 2013; Revelle & Zinberg, 2009). 

2.5.2. GMADE. The Process and Applications subtest of the GMADE (Pearson, 2004), a 
standardized assessment assessed students’ ability to comprehend mathematical language and 
concepts and apply relevant operations to solve word problems across multiple content areas 
(e.g., algebra, geometry, number and operations), was used as a general measure of problem 
solving. The Process and Applications subtest consists of 30 multiple-choice questions, including 
multiple-step problems and Process Problems. Each question is worth one point. The coefficient 
omega reliabilities for our sample were 0.67 for the pretest and 0.74 for the posttest. 

2.5.3. Mathematics MCA-III. This assessment measures students’ knowledge of concepts 
and skills in four strands: (a) number and operation, (b) algebra, (c) geometry and measurement, 
and (d) data analysis and probability. At seventh grade, the MCA is reported to have a marginal 
reliability (Dimitrov, 2003) of 0.93 for the online test and 0.91 for the paper test. Each subscale 
has an average internal consistency correlation with the total scaled score of 0.88 (Minnesota 
Department of Education, 2014). Reliabilities for this test for our sample were not computed 
because the responses of students to individual items were not available. Scaled mathematics 
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scores obtained in Spring 2013 served as a pretest and scaled scores for this test obtained in 
Spring 2014 served as a posttest. 

2.5.4. Treatment Acceptability Rating Scale-Revised (TARF-R , Reimers & Wacker, 
1988). Students rated four items related to diagrams (e.g., helped to organize information and 
understand how to solve problems) and five items related to problem solving procedures (e.g., 
helpful in solving ratio, proportion, and percent word problems; enjoyed solving word problems 
using the different strategies - cross multiplication, unit rate, equivalent fractions) on a l-to-4 
scale (4 = strongly agree to 1 = strongly disagree). Coefficient alpha was 0.84. 

2.5.5. Teacher Satisfaction Survey. The instrument included 22 items that focused on the 
benefits of: (a) SBI (i.e., the SBI curriculum will produce a lasting improvement in most 
students’ problem solving skills; the materials are helpful in implementing SBI), (b) diagrams 
(e.g., diagrams are effective in highlighting the underlying mathematical structure of problems 
and mediating problem solution), and (c) problem solving procedures (e.g., the focus on multiple 
solution strategies was effective in enhancing problem solving for most students). The items 
were evaluated on the same 0-to-4 scale as the TARF-R. For this sample, coefficient alpha was 
0.94. In addition, the instrument included space for teachers to write additional comments. 

2.6. Data Analysis 

Several multilevel (i.e. two-level, students within clusters) models with covariates at both 
levels were fitted using the HLM 6 software (Raudenbush, Bryk, & Congdon, 2004). These 
models were used to detennine if there were significant differences in the outcomes between 
classrooms of Cohort 1 and Cohort 2-teachers. Outcome variables were the PPS posttest, PPS 
delayed posttest, GMADE posttest, and the MCA mathematics posttest and each was analyzed 
separately to answer our research questions. All analyses were based on cross-sectional data. 
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For each outcome, the Level 1 (student) model contained four covariates: sex (0 = male, 
1 = female), and race (Black = 1, Hispanic = 1, Asian = 1, with White = 0 serving as the 
reference group), and for all outcomes except the MCA the pretest also served as a student-level 
control variable. All Level 1 variables were grand-mean centered. 

Level 2 (teacher/classroom) variables included the treatment variable (1 = SBI- 
experienced, 0 = SBI-novice) and variables capturing the percentage of students per classroom 
eligible for FRL and students receiving special education services. The variable percentage of 
ELL students was problematic because of the large number of classrooms with minimal number 
of students qualifying as ELLs: 62% of classrooms did not have any ELL students and another 
18% of classrooms had 10% or less students qualifying as ELL. This variable was not included 
in the multilevel models. Moreover, the coarse and discontinuous nature of the distributions of 
classroom percentages for the FRL and special education variables prompted us to rescale these 
variables to better capture the pattern in these data. Accordingly, these two variables were 
transformed to quartiles (i.e., 0 = lowest concentration of FRL students in a classroom, 3 = 
highest concentration; 0 = lowest concentration of special education students in a classroom, 3 = 
highest). The rescaled versions of these variables were used as classroom covariates. 

Slopes capturing the impact of the student background variables of sex and race on the 
outcome variables were found to be statistically equal across classrooms (i.e., variance = 0). 
Thus our analyses of the impact of cohort on student proportional problem solving are based on 
intercept-only models. Residuals from the fitted models were examined for evidence of 
violations to model assumptions of nonnality, homoscedasticity, and linearity and no major 
violations were found for any of the models. To control for compounding of Type I error rates 
we used the Dunn-Bonferroni correction in which an overall error rate (e.g., a = .15) was divided 
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among all tests of fixed effects linked to each outcome variable with no requirement that the 
error rate be divided equally. Accordingly, we assigned .05 to the test of the cohort effect 
because this was the most important effect in the model and divided the remaining .10 among 

.10 

tests of the remaining effects producing a = — = .0125 per test. For the analyses reported 

8 

below all available data were used meaning that sample sizes varied across analyses depending 
upon missing data patterns. 

3. Results 

A series of descriptive analyses were initially performed that included examining the 
correlations between all measures (see Table 2) as well as pre-existing differences between the 
SBI-experienced and SBI-novice teachers’ classrooms. Bivariate correlations between the PPS 
pretest, posttest, and delayed posttest were 0.67, 0.68, and 0.78, respectively. The correlation 
between the pretest and posttest on the GMADE was 0.63, and the correlation between sixth- 
grade and seventh-grade mathematics MCA scores was 0.94. Correlations ranged from 0.57 to 
0.68 across time points between the PPS and the GMADE tests and from 0.26 to 0.31 across time 
points between the PPS and the MCA tests. 

Table 3 provides descriptive statistics for the outcome measures by cohort. Differences 
between the Cohort groups on the PPS and GMADE pretests were d = 0.14 SD and d = 0.19, are 
statistically significant at a = .05 and favor students in Cohort 1 classrooms. It is not clear what 
the sources of these differences are, so we included these variables as covariates in the multilevel 
analyses to statistically adjust the outcomes for these differences and to establish baseline 
equivalence (see WWC, 2014). 
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3.1. Effects of PD 

To answer the first research question two-sample t -tests were conducted.to test cohort 
differences on the fidelity of implementation data. The Dunn-Bonferroni correction was used to 
control for compounding of Type 1 error by dividing an overall error rate (a = .15) among all 
statistical tests linked to each outcome variable. The mean total score across the seven items on 
the fidelity checklist was 14.06 (SD = 3.05) for Cohort 1 and 14.45 (SD = 3.26) for Cohort 2 out 
of a possible 21 points (higher scores are consistent with greater fidelity). Results indicated no 
statistically significant differences between cohorts on the total score and all individual items 
(Table 4). 

3.2. Cohort Effects 

Tables 5-8 report the results of fitting a multilevel model to the PPS posttest, PPS delayed 
posttest, GMADE, and MCA-III data, respectively that speak to the second research question. 
Results indicated that Cohort was not a significant predictor of PPS posttest scores, suggesting 
that students on average performed the same on the outcome regardless of whether their teacher 
was part of Cohort 1 or Cohort 2. The only statistically significant predictor among the student 
covariates was PPS pretest, with unit increases in this variable associated with an expected 
increase in PPS posttest of .73 (standardized effect = .06 SDs). The same pattern emerged for the 
PPS delayed posttest with the only significant predictor being the PPS pretest (.76, .65 SDs). 

Multilevel results for the GMADE posttest show that there were only two significant 
predictors. One indicated whether a student was Black (-1.05, -.06 SDs ) meaning that, with other 
predictors held constant, Black students tended to score 1.05 points lower on the GMADE 
posttest than White students. The significant pretest predictor (.62, .56 SDs ) indicates that each 
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one unit increase on the GMADE pretest was associated with a .62 expected increase on the 
GMADE posttest. 

Multilevel analyses of the MCA outcome also indicated that the only significant predictor 
was a student’s 6 th grade MCA mathematics score (.81, .89 SDs), meaning that each one unit 
increase in MCA 6 th grade mathematics scores was associated with a .81 point expected increase 
in posttest scores. 

3.3. Student and Teacher Acceptability of SBI 

To answer our third research question, students’ and teachers’ attitudes regarding SBI 
were compared (see Appendix B for a table of mean scores and standard deviations for student 
and teacher satisfaction data). The analyses (two-sample /-tests) of the TARF-R identified that 
student attitudes regarding SBI were significantly different {t = 4.01, df= 1791 ,P< .001) across 
cohorts. On a scale of 1 to 4, students in Cohort 1 classrooms had a mean score of 2.92 (SD = 
0.58) and students in Cohort 2 classrooms had a mean score of 2.81 (SD = 0.61). 

With regard to teachers’ attitudes, results were not significantly different (t = .043, df= 
71, p > .05). On a scale of 1 to 4, Cohort 1 teachers had a mean score of 3.30 (SD = 0.46) and 
Cohort 2 teachers had a mean score of 3.29 (SD = 0.41). Although Cohort 1 and Cohort 2 did 
not differ statistically in their attitudes toward SBI, descriptive evidence suggested that Cohort 2 
teachers were much more positive in their comments about the training, SBI program materials 
and intervention features than Cohort 1 teachers. Many Cohort 2 teachers commented that they 
liked the training and resources and that the SBI program was fun and effective in developing 
strong reasoning skills. These teachers especially liked the diagrams as they were helpful in 
organizing and explaining ideas. Also, they noted that the mathematics vocabulary (e.g., ratios as 
part-to-part and part-to-whole) as well as the problem solving procedure (i.e., identify the 
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problem type, using an appropriate diagram to represent the problem, solving, and checking the 
answer) and strategies (e.g., estimation) were useful and they would use them in their future 
teaching. 

Other features Cohort 2 teachers liked about the SBI program were the problem solving 
tasks, which were related to things that middle school students are interested in and the review 
lesson using a Jeopardy game format, wherein students read problems in different categories 
(e.g., sports, entertainment) and identified the problem type. In contrast, Cohort 1 teachers noted 
that they liked the SBI program because it focused on the quality of the problem over quantity. 
They found the percent of change diagram and the focus on multiple solution strategies were 
effective for most students. Furthermore, they noted that special education students struggled less 
with the SBI program as compared to their district adopted text. 

4. Discussion 

In the wake of the CCSS, fewer topics are explored and studied in depth at each grade 
level, such that in Grade 7, for example, proportionality is one topic that has merited focused 
instructional time. As school districts document whether students meet the new standards and 
monitor achievement trends, they have focused efforts on supporting teacher learning through 
selection of textbooks and associated instructional practices. Research underscores the 
complexities of implementing newly designed curriculum materials that support new standards 
(e.g., Obara & Sloan, 2009; Remillard, 2005). However, whether participating in a PD program 
and experience in implementing innovative practices influences student learning has not been 
empirically validated. 

In this study, we examined the relative effects of SBI implementation by experienced- 
and novice-teacher implementers. With regard to the first research question about the effects of 
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the PD training provided in this study for teachers who have prior experience with SBI vs. 
teachers with no prior experience with SBI on their classroom instructional practice (i.e., fidelity 
of treatment implementation), there was evidence of fidelity of implementation for both cohorts. 
This finding suggests that even though the amount of PD was significantly less than that 
recommended (30 to 100 hours) in the PD research (Blank & de las Alas, 2009; Yoon et al., 

2007) , the quality of the PD program may have had a substantial, positive influence on teachers’ 
implementation of the PD practices in the classroom (see Guskey & Yoon, 2009). We designed 
the professional development to incorporate important design elements such as (a) aligning what 
teachers learned in professional development with what occurred in the classroom (Timperley, 

2008) , (b) focusing on subject matter knowledge of proportional reasoning that students are 
expected to leam (Garet et al., 2001), (c) promoting and extending research-based instructional 
materials (Hiebert & Grouws, 2007), and (d) emphasizing personal learning and reflectiveness 
(Guskey, 1997). Our study suggests that teachers can use well-designed curriculum materials, 
along with guidance and structure during PD on how to use those materials, to positively 
influence their instructional practice (Perry & Lewis, 2011; Remillard, 2005). We created 
materials (e.g., diagrams to highlight the mathematical structure of problems) to explicitly 
support teaching and learning, which may explain the better curriculum implementation by both 
groups (Cohort 1 and Cohort 2 teachers). 

Regarding our second research question, namely, whether students of teachers who 
participated in the PD training and had prior experience with SBI demonstrated higher levels of 
proportional problem solving performance on an immediate posttest and on a retention test (10 
weeks later) than students of teachers who participated in the PD program and had no prior 
experience with SBI, we found no evidence of differences in these outcomes between SBI 
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experienced- or novice-teacher implemented classrooms. We also examined whether students of 
teachers who participated in the PD training and had prior experience with SBI demonstrated 
higher levels of overall mathematical problem solving performance and mathematics 
achievement as measured by standardized assessments than students of teachers who participated 
in the PD program and had no prior experience with SBI. Again, the results indicated no 
evidence to support advantaged outcomes for students in either experienced- or novice-teacher 
implemented’ classrooms. However, a particularly noteworthy result from the multilevel 
analyses is that the concentration of special education students in a classroom was not a 
significant predictor of any outcome. This provides evidence that SBI has closed the gap 
between students with and without disabilities. Given the previous contributions of SBI in 
special education (e.g., Jitendra, DiPipi, & Perron-Jones, 2002; Xin, Jitendra, & Deatline- 
Buchman, 2005), our results are encouraging in that the benefits extend to improving the 
learning of students with learning disabilities in inclusive classrooms. 

In sum, cohort was not a significant predictor of any of the outcomes, suggesting that 
student learning in proportional problem solving was on average the same regardless of whether 
a teacher was an SBI-experienced or SBI-novice implementer. Specifically, these findings 
suggest that teachers implementing SBI in their second year of the study did so in a fashion that 
did not enhance student learning of proportional problem solving relative to that of students in 
SBI-novice classrooms. On the other hand, these results may suggest the “holding power” of SBI 
in the sense that student learning did not drop off despite the SBI-experienced cohort not 
receiving any additional professional development in the second year. The results also suggest 
that the more proximal professional development experience of SBI novice teachers did not 
translate to greater student learning. 
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In short, there was minimal variation in student gains from differences in teacher 
experience with SBI. We believe these central findings are important and extend previous 
research on the effects of PD and SBI (Jitendra et ah, 2009, 2011, 2013, 2015). Unlike teachers 
in Fennema et al. (1996) who gained expertise as a result of participating in the 4-year 
longitudinal study, Cohort 1 teachers in the present study may not have gained the expertise with 
SBI to result in better implementation and student gains compared to Cohort 2 teachers. 
Alternately, teaching mathematics teachers (Cohort 1 and 2) how to facilitate active learning 
using instructional methods (e.g., SBI) that allow students to engage in problem solving in-depth 
and for longer periods of time (as in this study, which emphasized quality of problem solving 
over quantity), and to demonstrate understanding and explore multiple solution strategies 
translated to student proficiency in mathematics. Such practices align with quality PD that 
provides “explicit guidance or instruction in the models of teaching specified within curriculum 
materials” (Penuel, Gallagher, & Moorthy, 2011, p. 1000). Our results highlight the importance 
of quality professional development and carefully designed curriculum materials to support 
better curriculum implementation by both SBI- experienced and novice teacher implementers 
(e.g., Perry & Lewis, 2011) and similar effects on their students. 

Regarding our third research question about whether teachers and students who 
participated in the study reported higher levels of satisfaction with the treatment, we found no 
evidence of differences in the outcome for Cohort 1 and Cohort 2 teachers. One possible 
explanation for this finding is that the high levels of satisfaction we found based on the PD also 
leads to increased instructional mastery, which supports the finding of Sample McMeeking et al. 
(2012). However, the finding that students in Cohort 1 classrooms reported higher levels of 
satisfaction with the intervention than students in Cohort 2 classrooms suggests that teachers 
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who had experience with SBI may have a greater buy-in with SBI and motivated their students 
such that teacher experience with SBI does make a difference in enhancing student attitudes. 

4.1. Conclusions 

Our study is an attempt to contribute to the knowledge about what type of PD and how 
much experience with SBI is necessary for teachers to implement it with fidelity. Furthennore, 
this study deals with the importance of considering best practices such as using visual 
representations to highlight the underlying problem structure, problem solving and 
metacognition strategies, and developing procedural flexibility when solving proportion 
problems involving ratio, proportion, and percent. Such practices can help teachers all over the 
world to learn more about effectively supporting their students learning of ratios and 
proportional relationships. In this regard, we think the current study could possibly add three 
contributions to the extant knowledge base. 

The first contribution of our work is the extension of previous research involving 
characterizations of PD mathematics programs that affect teacher practice and student 
achievement (e.g., Carpenter et al., 1989; Perry & Lewis, 2011; Sample McMeeking et ah, 2012; 
Saxe et al., 2001). Our study suggests that our approach to integrate new knowledge for teachers 
around content and pedagogy (teaching a wide range of students) focused on proportional 
reasoning, along with time to work with other colleagues during PD in meaningful, guided ways 
might have provided an effective professional development experience to positively influence 
teacher practices as in previous studies (Carpenter et al., 1989; Perry & Lewis, 2011). Unlike the 
Carpenter et al. (1989) and Perry and Lewis (2011) studies that provided content-focused PD of 
an extended duration, the short PD (16 hr) in our study might not be a concern given that it 
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focused on using a well specified intervention (less open-ended than refonn-based strategies) and 
related set of curriculum materials for teaching a specific topic (proportional reasoning). 

Based on the results reported in this study, the second contribution is the extension of 
previous research involving SBI. Our study suggests that the SBI intervention found to be 
effective in enhancing student proportional problem solving performance (see Jitendra et al., 
2009, 2011, 2013, 2015) was also effective for students when teachers with different initial 
levels of experience participated in organized PD (see Sample McMeeking et al., 2012). Taking 
this into account, it seems clear that teacher-educators should support teachers by organizing 
content-focused PD on how to implement SBI that reflect opportunities to engage in problem 
solving at a deep level to promote understanding and differentiate instruction (i.e., a combination 
of explicit and open ended problem solving strategies) to support a broad range of learners 
(National Mathematics Advisory Panel, 2008). 

A third contribution of our study is the international relevance of our findings. 
International assessments of mathematics (e.g., TIMSS) have shown that on average students 
scored lower on more complex proportional problems (e.g., percent of change) than routine 
proportional problems. Considering that the mathematical competence of U.S. students is below 
the international average and the share of low achievers is at similar levels as students in several 
other countries (e.g., Lithuania, Slovak Republic, Italy, Luxembourg, see OECD, 2014), it might 
be that students in these countries would also benefit from effective instructional practices 
focused on critical content area (e.g., ratios and proportional relationships). Further research 
should be done to address whether the benefits of SBI would hold in other countries. 

Although this study can be considered another step toward understanding more about 
what is essential to successfully implement SBI (type and amount of PD, necessary expertise 
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with SBI) to affect student achievement, our results have to be interpreted carefully given the 
limitations of the study design and the need for future research to address questions arising from 
our work. This study was designed as the second year of a 2-year efficacy and sustainability 
study in which treatment teachers who participated during Year 1 continued in the study as 
experienced SBI implementers for Year 2, and control teachers in Year 1 received professional 
development for the first time and participated as novice SBI implementers in Year 2. The 
combination of teacher reassignment and teacher attrition produced a nonequivalent, quasi- 
experimental, cluster design with repeated measures, which implies that strong causal inferences 
are not warranted. Although conducting such longitudinal studies is difficult due to yearly 
changes in assignment of school personnel, we first need to replicate this SBI experience and the 
possible impact of a novelty effect in future research and include a group that receives the SBI 
materials but is not provided with PD. 

Second, it is important for future research to examine the sustainability effect of SBI 
when teachers implement SBI for longer than 2 years as in this study. A study in which teacher 
participants are randomly assigned to implement SBI for different durations (e.g., 2 years vs. 4 
years) is needed to examine whether there is a sustainability effect favoring longer duration of 
SBI implementation. Furthennore, this study is limited by the absence of a no-treatment 
comparison group in the design. However, recall that results of previous SBI studies provide 
evidence supporting the efficacy of SBI. Last, it is unclear whether the Year 2 effects would hold 
in a different geographic location. Our future work will investigate the Year 1 and Year 2 effects 
with a new cohort of teachers and students in schools within the U.S. in urban districts that 
include substantial diversity in the student population. 

Third, given that our focus on procedural flexibility is limited to task characteristics, 
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future research should also address subject or context characteristics (see Torbeyns & 
Verschaffel, 2015). At the same time, we think it would be necessary to analyze whether the 
types of strategies students used to solve proportional word problems would be effective in 
assessing students’ thinking, especially given that students in Cohort 1 classrooms reported 
higher levels of satisfaction with SBI than students in Cohort 2 classrooms. In order to 
understand to what extent our results could be due to the outcomes (correct answer to the 
problem) we used, further research that examines student outcomes in terms of strategy choices 
and use would be necessary. For example, a random sample of students in both cohorts could be 
individually interviewed as they solve some problems (a limitation of this study) to assess their 
strategy choice based on their individual mastery of different strategies as in Torbeyns and 
Verschaffel. 

Fourth, it would be interesting to explore whether the results obtained may be affected by 
variables such as the topics being studied. Future research might evaluate to what extent teachers 
can take the SBI approach, following PD in proportion content, and create their own practices to 
teach students to identify and solve other problem types (e.g., algebra and statistics). 

In sum, this study extends the literature on schema-based instruction and provides 
evidence that teachers with limited exposure to, and experience with, SBI can implement it with 
the same level of fidelity as SBI-experienced teacher implementers, and that students of SBI- 
novice teacher implementers perform as well as students of SBI-experienced teacher 
implementers. It suggests that when provided with guidance and structure, teachers can use well- 
designed instructional materials to strengthen student learning. 
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Table 1 

Participant Demographic Information by Cohort (SBI-Experienced, SBI-Novice) 




SBI- 

Experienced 

SBI 

- Novice 



n 

% 

n 

% 

Sex 

Female 

Teacher Information 

18 ' 58.06 

28 

66.67 


Male 

13 

41.94 

14 

33.33 

Race 

Asian 

1 

3.23 

1 

2.38 


Am. Indian 

1 

3.23 

0 

0.00 


Hispanic 

0 

0.00 

4 

9.52 


White 

29 

93.55 

37 

88.10 

Location 

Rural 

19 

61.29 

23 

54.76 


Suburban 

9 

29.03 

14 

33.33 


Urban 

3 

9.68 

5 

11.90 

Experience 3 

M(SD ) 

12.26 

(6.51) 

12.09 

(6.96) 

Methods Courses 

M(SD ) 

4.32 

(5.27) 

2.86 

(3.01) 

Math courses 

M(SD ) 

8.68 

(3.86) 

8.43 

(5.06) 

Age in years 

M(SD ) 

Student Information 

12.54 ' (0.42) 

12.59 

(0.36) 

Sex 

Female 

420 

50.36 

488 

48.27 


Male 

401 

48.08 

510 

50.45 

Race 

Asian 

42 

5.04 

71 

7.02 


Black 

69 

8.27 

73 

7.22 


Hispanic 

62 

7.43 

87 

8.61 


White 

648 

77.70 

767 

75.87 

FRL 

Yes 

300 

35.97 

433 

42.83 


No 

521 

62.47 

565 

55.89 

ELL 

Yes 

45 

5.40 

65 

6.43 


No 

776 

93.05 

933 

92.28 

SpEd 

Yes 

91 

10.91 

99 

9.79 


No 

730 

87.53 

899 

88.92 

Location 

Rural 

482 

57.79 

473 

46.79 


Suburban 

257 

30.82 

385 

38.08 


Urban 

95 

11.39 

153 

15.13 

Missing 


13 

1.56 

13 

1.29 

Note. Teacher data 

are based on 

the number of teachers who responded. 3 

= years experience 


teaching math; SBI = schema-based instruction; FRL = students eligible for free or reduced priced 
lunch; ELL = English language learner; SpEd = students receiving special education services; 
Student n = 1,845 but the total number of students varies slightly across variables because of missing 
data (all variables had missing data except Location). 
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Table 2 


Intercorrelations Among Student Variables 


Variables 

1 

2 

3 

4 

5 

6 7 

1. PPS Pretest 

- 






2. PPS Posttest 

.671 

- 





3. PPS Delayed Posttest 

.681 

.778 

- 




4. GMADE pretest 

.598 

.573 

.585 

- 



5. GMADE posttest 

.596 

.680 

.667 

.627 

- 


6. 6 th Grade Math MCA 

.293 

.299 

.305 

.312 

.286 

- 

7. 7 th Grade Math MCA 

.264 

.306 

.315 

.290 

.278 

.940 


Note. PPS = proportional problem solving; GMADE = Group Mathematics Assessment and 
Diagnostic Evaluation; MCA = Minnesota Comprehensive Assessments. All correlations 
significant, p < .001. 
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Table 3 

Descriptive Statistics for Student Measures by Cohort (SBI-Experienced, SBI-Novice) 



PPS pretest 

PPS posttest 

PPS delayed 

GMADE 

pretest 

GMADE 

posttest 

6 th grade math 
MCA 

7 th grade math 
MCA 

SBI - Experienced 

M 

14.45 

18.32 

18.07 

14.79 

16.57 

644.31 

744.84 

SD 

4.67 

5.27 

5.36 

4.08 

4.39 

20.66 

19.76 

N 

822 

815 

811 

821 

816 

777 

813 

Missing n 

12 

19 

23 

13 

18 

57 

21 

SBI - Novice 

M 

13.80 

17.93 

17.61 

14.01 

15.84 

647.84 

747.64 

SD 

4.68 

5.67 

5.61 

4.12 

4.67 

15.55 

12.9 

n 

994 

986 

954 

990 

973 

869 

895 

Missing n 

17 

25 

57 

21 

38 

142 

116 

Total 

M 

14.09 

18.10 

17.82 

14.36 

16.17 

646.17 

746.31 

SD 

4.69 

5.49 

5.50 

4.12 

4.56 

18.22 

16.58 

n 

1816 

1801 

1765 

1811 

1789 

1646 

1708 

Missing n 

29 

44 

80 

34 

56 

199 

137 


Note. SBI = schema-based instruction; PPS = proportional problem solving; GMADE = Group Mathematics Assessment and 
Diagnostic Evaluation; MCA = Minnesota Comprehensive Assessments. All test statistics are based on the total number of items 
correct, except the MCA Mathematics test, which is a scaled score. Student n =1,845 but sample size varies across variables because 
of missing data. 
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Table 4 

Fidelity of Implementation by Cohort (SBI-Experienced, SBI-Novice) 


Measure 

SBI 

n 

- Experienced 

M SD 

n 

SBI - Novice 

M 

SD 

t 

df 

P 

Fidelity of Implementation 

Identifies problem type 

31 

2.03 

0.98 

42 

2.05 

0.85 

-0.07 

71 

.943 

Problem similar/different 

31 

1.23 

1.09 

42 

0.98 

1.18 

0.92 

71 

.359 

Represents key infonnation 

31 

2.55 

0.51 

42 

2.55 

0.50 

0.01 

71 

.995 

Estimates solution 

31 

1.97 

1.20 

42 

2.05 

1.29 

-0.27 

71 

.788 

Multiple solution strategies 

31 

2.10 

0.83 

42 

2.40 

0.80 

-1.60 

71 

.114 

Provides complete solution 

31 

2.68 

0.75 

42 

2.57 

0.59 

0.68 

71 

.501 

Checks solution 

31 

1.52 

0.85 

42 

1.86 

0.90 

-1.64 

71 

.106 

Total score 

31 

14.06 

3.05 

42 

14.45 

3.26 

-0.52 

71 

.607 


Note. SBI = schema-based instruction. The means of the two cohorts on each variable were compared using a two-sample /-test none 

of which were significant using a = — = .0188 following the Dunn-Bonferonni procedure, p = /;-value; n = number of teachers 

8 

(classrooms) 
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Table 5 

HLM Results for PPS Posttest 


Fixed Effects 

B 

SE 

t 

df 

P 

Between-Student Model 






Intercept 

18.61 

0.403 

46.20 

69 

<.001 

Sex 

0.38 

0.186 

2.07 

1745 

.038 

Asian 

0.55 

0.426 

1.28 

1745 

.200 

Black 

-0.89 

0.389 

-2.28 

1745 

.023 

Hispanic 

-0.34 

0.364 

-0.93 

1745 

.354 

Pretest 

0.73 

0.022 

33.06 

1745 

<.001 

Between-Classroom Model 






Cohort 

-0.08 

0.372 

-0.22 

69 

.824 

SpEd 

-0.02 

0.167 

-0.15 

69 

.884 

FRL 

-0.27 

0.174 

-1.56 

69 

.124 

Random Effects 

VC 

SD 

2 

X 

df 

P 

Classroom 

1.71 

1.309 

276.64 

69 

<.001 

Student 

14.76 

3.841 





Note. FRL = percentage of students eligible for a free or reduced price lunch expressed in 
quartiles; SpEd = percentage of students receiving special education services expressed in 
quartiles; VC = variance component; student n = 1,754 nested within 73 teachers/classrooms. 
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Table 6 

HLM Results for PPS Delayed Posttest 


Fixed Effects 

B 

SE 

t 

df 

P 

Between-Student Model 






Intercept 

18.55 

0.361 

51.33 

68 

<.001 

Sex 

0.20 

0.187 

1.05 

1712 

.296 

Asian 

0.29 

0.426 

0.68 

1712 

.495 

Black 

-0.20 

0.401 

-0.50 

1712 

.614 

Hispanic 

-0.45 

0.370 

-1.21 

1712 

.229 

Pretest 

0.76 

0.022 

34.58 

1712 

<.001 

Between-Classroom Model 





Cohort 

-0.01 

0.334 

-0.04 

68 

.966 

SpEd 

-0.20 

0.150 

-1.33 

68 

.189 

FRL 

-0.31 

0.156 

-2.00 

68 

.049 

Random Effects 

VC 

SD 

2 

x 

df 

P 

Classroom 

1.25 

1.116 

216.72 

68 

<.001 

Student 

14.66 

3.829 




Note. FRL = percentage of students eligible 

for a free or reduced price lunch; SpEd 

= percentage 

of students receiving 

special education 

services 

expressed in 

quartiles; VC 

= variance 


component; student n = 1,721 nested within 72 classrooms/teachers. 
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Table 7 

HLM results for GMADE Posttest 


Fixed Effects 

B 

SE 

t 

df 

P 

Between-Student Model 






Intercept 

16.91 

0.344 

49.11 

69 

<.001 

Sex 

0.29 

0.161 

1.80 

1729 

.072 

Asian 

0.01 

0.373 

0.03 

1729 

.979 

Black 

-1.05 

0.338 

-3.12 

1729 

.002 

Hispanic 

-0.26 

0.314 

-0.83 

1729 

.408 

Pretest 

0.62 

0.021 

29.04 

1729 

<.001 

Between-Classroom Model 






Cohort 

0.15 

0.318 

0.47 

69 

.637 

SpEd 

-0.19 

0.142 

-1.37 

69 

.176 

FRL 

-0.35 

0.148 

-2.33 

69 

.023 

Random Effects 

VC 

SD 

2 

x 

df 

P 

Classroom 

1.24 

1.114 

266.19 

69 

<.001 

Student 

10.98 

3.314 





Note. FRL = percentage of students eligible for a free or reduced price lunch expressed in 
quartiles; SpEd = percentage of students receiving special education services; VC = variance 
component; student n = 1,738 nested within 73 teachers/classrooms. 
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Table 8 


HLM results for MCA Mathematics 


Fixed Effects 

B 

SE 

t 

df 

P 

Between-Student Model 






Intercept 

746.78 

0.789 

946.27 

66 

<.001 

Sex 

0.47 

0.257 

1.83 

1613 

.066 

Asian 

1.30 

0.601 

2.17 

1613 

.030 

Black 

-0.29 

0.544 

-0.54 

1613 

.591 

Hispanic 

-0.23 

0.519 

-0.45 

1613 

.653 

6 th Grade MCA 

0.81 

0.011 

74.91 

1613 

<.001 

Between-Classroom Model 






Cohort 

-0.45 

0.727 

-0.61 

66 

.542 

SpEd 

0.56 

0.328 

1.70 

66 

.094 

FRL 

-0.81 

0.339 

-2.40 

66 

.019 

Random Effects 

VC 

SD 

2 

x 

df 

P 

Classroom 

7.53 

2.743 

638.39 

66 

<.001 

Student 

25.93 

5.092 





Note. FRL = percentage of eligible for a free or reduced price lunch expressed in quartiles; SpEd 
= percentage of students receiving special education services expressed in quartiles; VC = 
variance component; student n = 1,622 nested within 70 teachers/classrooms. 
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